Using Scheduled Lambda Functions To Snapshot Volumes

Mon, 10 Sep 2018 10:22:40 -0400

One of the primary methods I use for backups when working inside AWS is the use of snapshots. They're straight forward in AWS and they're available in the AWS CLI which means they are scriptable. In the past I'd write a simple BASH script and schedule it with Cron but this is the year of Serverless so I decided to re-write the process with AWS Lambda. While I'm not going to post the exact code I'll cover a few of the tidbits below.

The premise of the script is that it will run on some schedule and describe all volumes in the account it's operating under. It will then filter out volumes that don't contain at least one tag that matches a regular expression. This RegEx will make sure we're working on volumes that contain the schedule the volume should have a snapshot taken on, the last time it was run, the name of the snapshot, etc.

Given my familiarity with the JavaScript AWS SDK and NodeJS in general I chose to use that framework to develop the lambda script. Since the JS SDK can return Promises I tend to make these types of lambdas a chain of promises, similar to this:


params = { ... }
EC2.describeVolumes(params).promise().then(data => {
    let promises = []
    // Magic goes here.
    return Promise.all(promises);
})
.then (data => {
    let promises = []
    // Even more magic goes here.
    return Promise.all(promises);
})
.then(data => {
    callback(null, data);
})
.catch(err => {
    callback(err);
})

Fill in the blanks and you have a pretty handy Lambda function that can work with not only the EC2 API but any AWS service that returns promises through the JS SDK. One example is the SNS service. I use this one to send out emails when the snapshot has been preformed but you could just as easily send them out in the catch() to report errors instead of success.

One big thing to emphasize though is that most of the configuration happens on the volumes themselves. In my particular use case the volumes have snapshots created varying schedules. This requires setting up multiple cron jobs or lambdas with different configurations. However, with this new lambda each volume can have its own configuration that can be updated with me having to re-deploy the lambda with a new configuration itself.

Lambda | Joshua Hull

Using Scheduled Lambda Functions To Snapshot Volumes